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ABSTRACT 

Problem-centered project work, involving groups who 
work and learn col laborat ively , is an important form of instructional 
organization. This is especially true for students learning about the 
design and development of multimedia learning materials, since 
working as members of design teams will be an important part of the 
professional futures of many of the students. At the University of 
Twente in The Netherlands, students in the Faculty of Educational 
Science and Technology have an initial experience in the 
multimedia-design process in a first year required course. Covering 
two cycles of the course, this paper describes the instructional 
design of the course itself and illustrates how information 
technology is both studied and used as the learning environment in 
the course. It shows how the World Wide Web (WWW) provides an 
integrated setting for the students, so they can experience as 
learners the sorts of environments they are studying and designing in 
their groups. Emphasis is placed on: the integration of process and 
product; evaluation as a key unifying process throughout the course; 
the use of "productive communication" (semi-structured messages 
entered on the class web site); learning how to learn; and the 
implications of the new roles for both students and instructors. An 
appendix includes student evaluation feedback data. (Contains 20 
references.) (Author/SWC) 
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INFORMATION TECHNOLOGY EDUCATION IN A 
COOPERATIVE ENVIRONMENT: DESIGN AND EVALUATION 



Betty Collis* and Jeroen Breman** 

*University of Twente, The Netherlands, collis@edte.utwente 
** Pennsylvania State University, jxb66@psu.edu 



Abstract 

Problem-centered project work , involving groups who work and learn collaboratively , is an 
important form of instructional organization. This is especially so for students learning about the 
design and development of multimedia learning materials ; in that working as members of design teams 
will be an important part of the professional futures of many of the students. At the University of 
Twente in The Netherlands , students in the Faculty of Educational Science and Technology have an 
initial experience in the multimedia-design process as an integral part of a first-year required course. 
In this paper , we identify key issues in the instructional design of the course itself illustrate how 
information-technology is not only studied in the course but also used as the learning environment, 
and in particular show how the WWW provides an integrated setting for the students , so that they can 
experience as learners the sorts of environments they are studying about and designing in their groups. 
An extensive evaluation of the first cycle of the course (September 1995-June 1996) was carried out, 
and its results integrated into the 1996-97 cycle. In particular, stress is being given to the integration 
of process and product, to evaluation as a key unifying process throughout the course, on the use of 
“productive communication ”, our emphasis on learning how to learn, and on the implications of the 
new roles for both students and instructors. In each of these categories the course ISM-1 WWW site 
plays a critical role in the “pedagogical engineering” of the course which is occurring. The course 
site can be accessed at http://www.tMtwente.nl/ism/isml -96/home. htm 



Group-Based Cooperative Learning for Multimedia Design Education 

Although the importance of instructional methods that foster learner self-reliance and that 
support cooperative learning have been long established, their combination in a course for problem- 
oriented multimedia design education is less familiar. Learner self-reliance has many aspects, including 
self-awareness, “actively willing to seek feedback from others and able to give constructive feedback”; 
skill in action planning and implementation, such as the ability to organize time effectively, and to 
monitor and evaluate progress against specific criteria; and the development of the habit of reflection, 
learning from one’s own experiences as well as those of others (Hawkins & Winter, 1996). Contributing 
effectively to group-based cooperative learning situations involving working together to design, 
develop, and evaluate a solution to a complex problem, requires all not only self-reliance skills on the 
part of each individual group member, but also success with additional aspects such as consensus about 
the group’s goals, individual responsibility toward the communal attainment of those goals, efficient 
procedures for group self-management, and group cohesiveness and internal positive spirit (Slavin, 
1990). Such problem-centered, cooperative instructional orientations are seen as are seen as critical for 
the development of life-long learners (Casey & Tucker, 1994), and perhaps one domain where life-long 
learning will be most important is that of the educational technologist. The rapid emergence of new 
forms of media and new communication channels is leading to an explosion of new forms of media 
products, including environments and tools as well as structured learning resources for hyperlinked 
multimedia distributed networks (Collis, 1996). The educational technologist (or instructional 
technologist) will be increasingly challenged to provide professional leadership in rapidly changing 
technical and pedagogical environments, and thus life-long learning will be a job requirement 
(Gustafson, 1993). 

There are some examples in the literature of descriptions of courses for prospective 
educational technologists that are designed around this kind of future-looking, group-based, project 
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orientation for interactive multimedia design and production. For example, Liu (1996) at the University 
of Texas at Austin uses a collaborative learning approach to “simulate real-world multimedia 
production with her graduate students and notes that “the authentic aspect of the learning experience 
motivated many students because they perceived it as helpful to get them better prepared for the job 
market. The collaboration and interaction among groups enhanced students’ understanding of the 
multimedia technology” (p. 787). Liu and Rutledge (1996) also extend this approach with much 
younger learners, secondary students, also with positive results in terms of both process and product. 
But regardless of the maturity of the learners, the approach is challenging to implement in practice, 
particularly for the instructor, who among many responsibilities must monitor appropriate time 
management by the learners, find an efficient and effective method for feedback and motivation, and 
foster knowledge construction as well as product production (for example, see Liu & Rutledge, p. 400). 



General Description of the Course “ISM-1”, Cycle 1 (1995-1996), and Cycle 2 
(1996-1997) 

At the University of Twente, students in the Faculty of Educational Science and Technology 
work within a problem-solving framework throughout their study. In fact, the Twente approach to 
educational technology is that of “an engineering approach to educational problems’. Within the 
Faculty, one department specializes in the design, development, and evaluation of educational media 
products, what the Dutch call “instrumentatietechnologie”, or literally, the technology of learning 
instruments. This department is known internally as “ISM” and thus its required course for first-year 
students is called “ISM-1”. The course is regularly refreshed and revised, to reflect the rapidly changing 
nature of educational media. In the previous few years, considerable attention has been given to 
moving from older forms of media such as audio-slide presentations, text- and graphics- based 
computer simulations and tutorials, and interactive-video products to computer-based multimedia. In 
1996, the increasing educational use of distributed networks to access multimedia resources, 
particularly via WWW protocols, means that instrumentation for these sorts of situations needs to 
become part of the students’ experience. In particular, we have three general goals for our students: that 
they learn about educational media, its implementation possibilities and its effects in its full range of 
forms; that they gradually become competent as professionals in the overall design and life-cycle 
process of learning materials; and that they base this learning, not only on theory but also on practice. 
Also, the staff has an additional goal: that we practice what we preach, and therefore that students gain 
critical insights into educational media through actually learning with such media themselves. 

The course “ISM-1” is the first initiation of the students to the field of educational media and 
to the particular practice and philosophy of the Department ISM. Following is a brief description of the 
course. In particular, we refer to the two most-recent cycles of the course, that of 1995-96 (referred to 
as Cycle 1) and the current 1996-97 (referred to as Cycle 2). 

General goals. 

There are four major, intertwined sets of goals for the course, relating to its: 

-Content and theoretical framework: To introduce students to major forms of educational 
instrumentation, to a model for the lifecycle of media products, and to terminology and 
concepts relevant to each of these phases as well as to different forms of educational 
instrumentation . 

-Practical experiences with software tools and media-related equipment: To introduce 
students to a broad range of professional tools and equipment for the design and 
development of educational instrumention, and to scaffold students in their use of these 
resources in their own project experiences. 

-Group-related goals: Critical to the course is the experience of working as a group, 
toward the solution of a realistic problem. The “Jigsaw Method” (Aronson, Blaney, 
Stephen, Sikes, & Snapp, 1978) is used in the course. In the Jigsaw Method, every 
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student has his or her own unique responsibility to the group; like a piece in a jigsaw 
puzzle, the result can not be whole without all these unique pieces fitting well together. 

-Individual goals related to the process of becoming a professional: The research from 
groupwork emphases not only the importance of group goals and group interrelationships 
in social constructivism, but also the importance of the individual in his or her own 
learning and development (Jonassen, 1995). In ISM-1, important goals relate to the 
individual learner him- or herself. Learning how to learn, learning to take self- 
responsibility for one’s own study habits, learning how to be an effective self-evaluator, 
learning how to ask questions and how to find help after one has used one’s own 
resources for answers; all are important. 

Course organization 

The course is organized around three problem settings, each of which motivates a group 
project extending over a trimester (12 weeks). Each project in turn is based on a realistic problem. 
Students are organized in groups of eight or nine, where each group has a Manager, and each pair of 
students has a “specialism” which they must bring to the common solution of the problem that the group 
is confronting. . Each group is responsible for contributing a component of an overall solution to the 
given problem. For example, the problem for the first project of ISM-1, 1996-97 cycle, was to help 
first-year students in the faculty (in fact, themselves), to find information about different problems 
which they might encounter as students new to the university, problems with housing, finances, personal 
relations, health, study skills, choice of study, emotions and stress, and to have a convenient way to 
make contact with appropriate persons who can help them further. The WWW was chosen as an 
appropriate instrumentation form for such a communication and information system. Each group was 
responsible for designing and constructing a suite of WWW pages relating to one of the problem areas, 
and together all eight sets of WWW pages would comprise the overall solution. 

Time commitment 

Each group is expected to work together for four hours per week, during a scheduled time but 
generally without an instructor present. In addition, each student individually is responsible for staying 
on tempo with the study materials of the course, approximately two hours per week, and for attending 
all of the five scheduled lecture sessions per project. The total over the full year should relate to 240 
hours of participation time. 

Determining the course mark. 

The final mark for the course consists of the average of the three group-project marks (with in 
general all members of the group receiving the same mark) and the scores on three written tests, one 
after each project. Students must have at least a passing mark in each of these for the overall mark to be 
averaged and awarded. 

Integrating the course within an electronic environment. 

In the first cycle of the course, the FirstClass computer conferencing environment was used 
not only for all course communications but also for course and project organisation and for lecture 
notes. There was a separate textbook as well as various sets of handouts for the students which they 
were expected to keep in a binder. In the second cycle, taking insights from an extensive evaluation 
carried out during the 1995-96 cycle (Breman & Carleer, 1996) all course resources have been 
redesigned within a single WWW-based course environment. Instead of a textbook, all reading 
materials are included within this environment. This WWW environment is discussed extensively 
further in this paper. 

Evaluation study and results 

Rather than giving more overall description of the ISM-1 course, we will instead focus on six 
major attentions points emerging during the 1995-96 cycle and its evaluation, and describe the 
evolution of the 1996-97 cycle of the course relative to these attention points. The evaluation was 
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commissioned by the Faculty because of the innovative aspects of the course for first-year students. 
Could these young students work without direct supervision? Could they handle group work, combined 
with self-responsibility? If the emphasis was on process, rather than product, what would be the norms 
for satisfactory achievement? An external evaluator was appointed to work closely with the Course 
Team throughout the 1995-96 year, as an observer and also as someone offering on-going comments 
and impressions. The evaluator also observed and interviewed students as they worked on the projects, 
interacted on a personal basis with different groups of the students in the course throughout the year, 
and prepared a written analysis after each of the three projects as well as a final report (Breman & 
Carleer, 1996). 

The final conclusions of the evaluation study reflected what the student achievements had 
already shown over the full 1995-96 cycle: That the group approach worked well, that the tasks and 
assignments were successfully completed with much motivation and enthusiasm (every student but one 
completed the course successfully), and that all involved in the course worked together responsively 
and creatively to learn from each experience in the course and to adapt and evolve the course 
accordingly. Similar results have been found from the first project of the 1996-97 cycle (See the 
appendix ). 

Given this overall, the remainder of the paper discusses major aspects of the course, and its 
WWW-based learning environment, in more detail. 



Relating Theory To Practice: The Integrated Course Site 

In 1996-97, the entire course is integrated within one WWW site. This site can be visited at: 

http://www.to.utwente.nl/ism/isml-96/home.htm 

The site for Project l consisted of many hundred pages, all developed by the Course Team (sometimes 
with input from the students themselves, as will be described later). For a sense of consistency 
throughout the site and to help the students realize when they were still within the course site and when 
they were in an external link, a course logo reappears on most internal pages. Figure 1 shows the logo. 




Instrumentat ie 
Technologie |] 

FACULTY OF EDUCATIONAL SCIENCE AND TECHNOLOGY 

imnEBsrrtaevimtm 



Figure I. Logo to identity internal pages in the ISM-1 course site. 



The “Week-by-Week” and “Study” Centres. 

The integration of theory and practice is always a challenge, particularly in project-oriented 
courses. Activities cannot be metered out as practice exercises following a section of theory, as can be 
the case in a mathematics course, but must occur as the higher-level culmination of many different 
insights. Theory must be translated and applied to practice. The colleges and study materials in ISM-1 
are meant to make this translation as explicit as possible. Thus, based on the experiences of the first 
cycle of ISM-1 as well as research occurring at TO and elsewhere, the decision was made to integrate 
all aspects of ISM- 1, theory and practical, within one hyperlinked environment where explicit linkages 
can be made between weekly group work and weekly study materials. Figures 2 and 3 show the “Week- 
by-Week Centre” and the “Study Centre” of the ISM-1 Course Site to illustrate how the materials are 
presented to the students. Within each link of these centres, explicit hyperlinking occurs between the 
concepts in the Study Materials and College notes, and the assignments for the group activities for the 
week. 
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Figure2. The “Week-by-Week Centre ”, linking to study materials and assignments 
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Figure 3. The "Study Centre", giving access to the theoretical materials of the course 



From bound to on-line textbook. 

In the first cycle of ISM-1, there were some complaints by the students about the textbook and 
its relevance, and about the lectures and their relevance. The Course Team was also not pleased with 
the fit of the textbook to the projects, even though it was the best textbook that could be found. The 
unique aspects of the ISM-1 course mean that no single externally produced textbook is likely to be 
available, appropriate for the level of the Dutch first-year students and capable of being closely related 
to each week of their group activities. Thus the decision was made for the second cycle of ISM-1 that 
the Course Team would write the theoretical materials for the course, in a form appropriate to both the 
group activities and also as a example to the students of how hyperlinked study materials (about which 
they will study in other courses in ISM and TO) can be used as learning resources. 

The Course Team sees these hyperlinked study materials as having the following benefits: 

• Capable of being fine-tuned to respond to each week’s group experiences; thus to serve as both 
theoretical material and feedback from the Course Team to the class as a whole 

• Capable of being linked to multimedia examples, both of student work and from outside WWW 
sites 

• Capable of including self-testing and evaluation questions via JavaScript and CG-I forms, to 
stimulate interactivity while self-studying and to collect student responses to evaluative questions 

• Capable of showing, as explicitly as possible, how the colleges, the study materials, the course 
feedback, and the group activities all are intended to be interrelated 

Evaluation work in Scotland (Brown, Doughty, Draper, Henderson, & McAteer; 1996) 
indicates that the effective integration of resources into higher-education courses must be done in such a 
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way that students are motivated to make use of the resources, that the resources should be easy to 
access, that students must know the resources are available to them, and that the instructors must 
communicate to the students why the resources are relevant. Students are practical, and “may ignore 
anything not felt to be essential.. .use is often related to the time available to use them and the problems 
associated with doing so”(p. 111). By integrating the study materials with the course communication, 
the marking in the course, and the project work, all within the same WWW-based environment, and 
emphasizing the value of these study materials to the weekly activities, we hope to increase the 
likelihood of careful use and study of embedded resources. 



Integration Of Process And Product 

Important to the course is the integration of support for the group processes of the course, not 
only with the study materials but also with the products that the students design and produce. The 
WWW environment allows this integration to take place, at least for some of the products that the 
students will produce. Figure 4 shows the “Group Centre” environment for the course from the Course 
WWW Site for Project 1 of 1996-97. From this page, students can find the composition of their groups 
and their specific tasks, the managers of the groups were given structured group-report forms to 
complete on various occasions, the self-evaluation of the group relating to their own products was 
always available for reference, and the products themselves could be seen and evaluated by the Course 
Team and by the students in all of the other groups, as regular evaluation activities throughout the 
project. We discuss this process more in the next section. 
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Figure4. The “ Group Centre", showing links to Manager inputs, self-evaluation comments by each 
group, and giving access to the current stage of each group ’s products. 
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Evaluation As A Key Process 

Not only because of important of developing self-evaluation skills in students as part of their 
growth toward more self-reliant learning (Hawkins & Winter, 1994) but also because of the more- 
general importance of on-going evaluation in both the multimedia lifecycle and the lifecycle of a 
course, we are working hard to transform evaluation from something given out at the end of a process 
by the instructors to something ongoing throughout the project, and for which the students themselves 
have the major responsibility. 

Formative evaluation as a key to integrating theory and practice. 

To accomplish this more effectively in the 96-97 cycle of the course we are utilizing a number 
of methods (Collis & Meeuwsen, 1997). One of these which is emerging as particularly versatile is that 
of making use of the capabilities of the WWW environment to allow us to embed CG-I forms to 
structure group self-evaluation and peer evaluations, and to base these evaluations not only on the 
concepts about formative evaluation developed in the Study Materials, but also to make the criteria for 
the evaluation be the set of “design guidelines” for the course. These design guidelines are developed 
week by week in the Study Materials, illustrated with examples from both external links and during the 
colleges, and used repeatedly as the criteria for formative evaluations of the group products. The design 
guidelines are accumulated in the Resource Centre of the Course Site. Twenty of them were developed 
during Project 1. The Managers were directed during Week 44 to lead their groups in self-evaluation of 
their sites, and to submit a report, via CG-I forms, in which they selected two of the guidelines that they 
felt were well met by their work and two of the guidelines for which improvement should be made. 
These self-evaluation reports became part of the Course Site (see Figure 4). 

Peer evaluation . Following the self-evaluation, came peer evaluation. Figure 5 shows an excerpt from 
the Assignment materials for Week 45 of the course in which the formula to be used for peer (and later, 
Course Team) evaluation was presented based on the 20 design guidelines and the specialist 
responsibilities. 
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During Week 45, you need to (1) finish your sites, and (b) carry out a formative evaluation of each 
other's sites. 

You will still have time for last improvements in Week 46, making use of the formative evaluation 
results which you will receive from the other groups during this week (Week 45). 



Evaluate the sites of the other groups... 

1. Work with a partner; go through the sites of the other groups, from the homepage column of the 
Group Centre . Make notes to yourselves on the sites, thinking about the following categones and 
Design Guidelines: 
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Figure 5. Relating the design guidelines to specialist responsibilities and to the peer evaluation 

Figure 6 shows the peer evaluation form, available via the WWW site as part of the Assignment for 
Week 45 of the course. The instructions for the week were that the Manager of each group was to 
organize his or her group so that members of the group evaluated each of the other seven group sites, 
and filled in the formative evaluation of that site. These forms were submitted (by a click on the submit 
button), sent to the student assistant who in turn forwarded them to the Manager of the group being 
evaluated and to the Course Team. It was in turn the Manager’s task to discuss the peer evaluation 
comments with his or her group and to lead the decision making about how to revise their sites based on 
these evaluative comments. 
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Figure 6. CG-I form for peer evaluation (and later for Course Team evaluation) 

Evaluation as study material for the examination 

But the integrative role of formative evaluation throughout the course does not stop with the 
peer evaluation. The Course Team told the students very clearly that the final evaluation of their sites 
would be based on not only the Course Team’s use of the same evaluation forms, but also that the peer 
evaluation would be taken into account. Finally, the examination to follow the first project contained a 
major section relating to the evaluation process and its results: asking the students to reflect on points of 
agreement and disagreement between their own self-evaluations of their work, the evaluations of their 
peers, and the Course Team evaluations. Students were warned to expect this question; all feedback 
from the Course Team was placed in the Course Site (in the Group Centre) the day after the final 
presentations of the projects, so that the integration of theory and practice, through the on-going process 
of evaluation based on design guidelines, could be reflected in the written test for the project. 

Through these processes, we believe we are making significant progress toward not only 
integrating the theory and practice of the course but also toward helping students develop insight and 
self-responsibility for their own evaluation. Evaluation should be seen as part of the on-going process 
not only of the course but working as a design professional. (A walk-through of all of the above 
evaluation-related aspects of the course can be followed by the WWW-site “guided tour” available at 
Collis & Meeuwsen, 1997). 
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Communication: Multiple Forms And Purposes 

Students can see and hear from all course team members at the lecture sessions and can of 
course present questions but in a room of 85 students question-asking is only something that a few 
students per session can much contribute to. The Study Materials can be seen as a different form of one- 
to-many presentation, containing not only conceptual matter but occasionally personal reflection from 
the Course Team. Less-formal messages to the whole class are also important. The major example of 
these is the “First Stop” page of the Course Site, indicated by an ever-present icon (a lightening bolt) in 
the top left-hand corner of the navigation frame. The students know that it is their responsibility to 
check First Stop first thing on Monday morning of each week. There they find an overview of the week: 
what is new in the Course Site, what they should read and do for the week. This message is kept brief; 
its intention is to point the way to other messages that give detailed communication about weekly 
activities. 

“Productive communication” 

An major innovation in ISM-1 1996-97 compared to the 1995-96 cycle is the use of CG-I 
forms for “productive communication.” With a CG-I form, input expected from the student can be 
highly structured (see the peer evaluation form in Figure 6) or can still be as open as an e-mail message. 
This allows us to indicate clearly to the students what input is expected, and to make it as easy as 
possible for them to enter this input in a consistent fashion. The student simply types directly into the 
Course Site. During the 1995-96 cycle of ISM-1, the inability to create structured input forms in the 
computer conferencing environment led to two less-than-ideal situations: Either students replied via an 
unstructured message, and thus were not always sure of how much was expected of them in responding, 
or the Course Team had to switch to separate paper forms when structured input was critical to 
providing guidance. Also, the flood of communication coming from the students, with many different 
reports and summaries, led to an unmanageable amount of mail for the Course Team to effectively read, 
handle, forward, and archive (there were over 3,000 e-mail messages of various sorts after the total 
1995-96 cycle). 

But it is not the ease and clarity of expectation that makes CG-I-form communication most 
important in ISM-1. It is the fact that students’ communication can be clearly seen to serve a purpose in 
the Course Site, in fact, to become part of the study material for the course. (We have already 
mentioned how this is critical to the on-going evaluation processes in the course). This “productive 
communication” occurs by entering the content of the students’ CG-I-input into HTML form, linking it 
to the Course Site, and then making use of it in the instructional activities of the following week or 
weeks. In this way, students can see that entering their input, on time, has a function to the group as a 
whole, because it becomes part of the learning material for the group. It also helps students to develop 
their self-evaluation skills, in that seeing what other students have entered and comparing it to one’s 
own entry can be a stimulus to self-monitoring. 

An example of the use of productive communication within ISM-1 can be seen from the 
Specialist Centre page (Figure 7). After the Specialist Sessions for each of the roles in Project 1, 
students were required to input, via a CG-I form, a brief summary of the main points to remember from 
the session. These summaries were merged into one HTML page, and then were always available from 
the Specialist Centre page. When students later had a question relating to any of the specialist roles, 
they could turn to the “Specialists’ Summaries” as well as to the “Bookmarks” provided to them by the 
Course Team, as their first line of help. Figure 8 shows a portion of the input from the HTML 
specialists, available from the Specialist Centre page. Da Bormida, Donzellini, and Ponta (1996) also 
have commented on the importance of students becoming involved in the creation of their own learning 
material, “under properly defined rules, students are at the same time, users, producers, and 
evaluators. ..’’(p. 160). The WWW environment in ISM-1 makes this productive communication 
convenient, as well as accessible to all. 
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Figure 7. The Specialist Centre, with learning resources for the various roles, and also with links to 
CG-I forms for Specialist reports. 
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Summary group 4 by Annechien Langevoord & Marianne 
Polderdijk 

INHOUD: ' 

1. Het principe van HTML 

2. Enkele verwijzmgen naar Web-pagina's 

3. lets over BBedit 

ad 1. HTML is de afkorting van HyperText Markup Language. Hiermee kun je door het gebruiken 
van vastgelegde codes eenpagina-opzetmaken. Door onder 'View' de optie 'document source' te 
kiezen kun je de pagina-opzet van elke pagina bekijken. Hier volgen enkele voorbeelden van codes: 
TITLE titel invullen /TITLE, B tekst die vet gedrukt moet worden /B. Zie voor nog meer 
HTML-codes verwijzing a. bij 2. 

ad 2. Enkele verwijzingen naar W eb -p agina' s : 

a. Bare Bones Guide to HTML 

b. HTML: Hypertext Markup Language 

c. InfoServices HTML info 

d. Infinet Colors 

ad 3. BBedit is het programma waar de HTML-codes in verwerkt zijn zodat je eenvoudig een opzet 
kunt maken. Je komt in BBedit door onder het Apple-menu Editors te kiezen en daarin BBedit lite 
3.5. 1 aan te klikken. Voor een stan- daard codering voor het beginnen van een pagina kies je onder 




Document Done 






Figure 8. Summary of the specialist reports, available as a study aid for HTML coding. 

Toward Self-Responsibility: Learning How To Learn 

An important part of the philosophy of ISM-1 is scaffolding the students toward self- 
responsibility . The example shown in Figure 8 is typical of this philosophy. If students have a question, 
they must learn a series of strategies to find help themselves. The Course Team has the responsibility to 
make as much help available as is useful, such as seen in the Bookmarks and Specialist Summaries in 
the Specialist Centre page (Figures 7 and 8). But the students must learn to look for help in the 
Specialist Centre, and in other places in the Course Site. After this, the students must learn to ask each 
other for help; this is part of the peer teaching philosophy of the course and consistent with Vigotsky’s 
theories of peer support. Students also have available to them examples of other students’ work 
(through the Group Centre page, for example), and should learn to look at those examples for help. 
After this, the Communication Centre is always available, and a message sent to the Course Team 
members will be answered within the day. But of course, unaccountable problems do arise, often related 
to the network or something like a room being unexpectedly locked. Students must learn how to classify 
their problems, and how to identify which require immediate human intervention (unlocking a door) 
compared with those that require patience and regular checking (waiting when the network is down). 

Despite the efforts that were made in the 1995-96 cycle of ISM-1, the comment was expressed 
in the cycle 1 evaluation report that students needed (or wanted) a clearer sense of where to get help. 
Thus the 1996-97 Course Site is being carefully designed to make adequate help clearly available. 
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However, providing adequate on-line help, and supporting students in their growth toward being self- 
reliant and good peer supports for each other, is not a simple task, and will continue to be an action area 
for the Course Team. One idea we hope to develop is a question-and-answer component in the Course 
Site, where we can build on students’ questions and evolve these into an “answer web” (Smeaton & 
Neilson, 1994) that students can consult, and to which they can contribute. Another focus is to continue 
to try to help the Managers develop more-effective managerial skills, not an easy task for students just 
out of secondary school (or for professionals, as well; see McDaniel and Liu, 1996, for a good analysis 
of the skills and insights needed for good management of multimedia development projects). In the 
“Group Planning Pages” for the second project of the 1996-97 cycle of ISM-1, we are now using a 
WWW-based approach to a project-management tool to help Managers more in their tasks. 

The Evolution Continues 

The course ISM-1 is a learning experience for all involved, and thus we can identify many 
areas in which we will continue to pay particular attention even after the 1996-97 cycle. A research 
project is already underway focusing on helping students become better at managing their own work 
and helping the instructors to have adequate information about the students’ work so that appropriate 
feedback can be given “just in time” (Van der Veen, 1996). Another research project will begin in 1997 
in which scaffolding will receive major attention: how can scaffolding be effectively incorporated, and 
withdrawn, for students working more and more independently with WWW-based environments, such 
as the group project work in the course ISM- 1? (Collis & Verhagen, 1996). Another important focus is 
peer teaching: How can we help the students to help each other in increasingly effective ways? (The 
“intelligent coach” to support the Jigsaw Method in teams of computer science students working 
together on complex programming tasks which has been developed by McManus and Aiken, 1995, is an 
example we wish to explore). The long-standing issue of media selection is also important to us: The 
integrated WWW site offers many benefits compared to non-electronically linked course materials, but 
the important benefits of other media, such as printed text, need to be also considered. Perhaps not 
everything should be in the WWW site, even if it is digitally possible, but by what criteria do we 
decide? 



A particular focus is that of integrating multimedia resources in the course site. As one aspect, 
how can we, as Da Bormida and his colleagues suggest (1996), make increasingly effective use of the 
students’ own work as examples within the Course Site? Partly this is a technical challenge. We are 
already exploring the integration of CD-ROM materials with the Course WWW site during the 1996-97 
cycle in order to better utilize the video products made by the Cycle 1 cohort. In addition, in order to 
continue our emphasis on linking theory and practice, we would like to be able to link videotaped 
fragments of the lectures to appropriate points in the students’ group work and self-study. To do this, 
we are planning for a video-on-demand component to eventually become part of the course site. 
Integration of multimedia resources, either from the students’ work or from videotaped fragments or the 
lecture sessions, is by no means only a technical issue. The instructional issues and design issues 
involved in selecting appropriate video segments of appropriate lengths, indexing them effectively, and 
pointing out to the students how to apply those examples to their own work will occupy our attention, as 
designers, instructors, and researchers, for many years to come (Verhagen & Blanke, 1996). 
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Appendix 

Evaluation feedback from students, Project 1 of 1996-97 cycle of the course “ISM-1” 

Note: The evaluation results were submitted by the students in the 1996-97 cycle of the course (n =85) 
via a CG-I script within the course WWW site early in the first project and after the completion on the 
first project, as a paper-based questionnaire completed during the first examination. For each item, there 
was a five-point scale, with “1” labelled as “strongly disagree”, “2” as “disagree somewhat”, “3” as 
neutral, “4” as agree, and “5” as strongly disagree. The following compares some of the responses at 
these two time periods. 



Questions (n = 80 students responding, out of 88) 


Near the start of 
Project 1 (Sept. 96), 
median score 


After completion of 
Project 1 (Dec 96), 
median score 


1. The communication between the group members is 
sufficient. 


4 


4 


2. My group has maintained a good division of tasks. 


4 


4 


3. The efforts of each individual group member on the 
project work are comparable. 


3 


3 


4. 1 was able to work by myself on my tasks without help. 


4 


4 


5. The support from the staff members is sufficient. 


4 


4 


6. 1 enjoy working on ISM-1. 


4 


5 


7. I like the specialist tasks that I have in my group. 


4 


4 


8. I like the topic our group is working on. 


4 


4 


9. I am satisfied with the learning effect of my work so 
far. 


4 


4 
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